WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 5 
H04N 5/76, 5/91, 5/222 



Al 



(11) International Publication Number: WO 94/27404 

(43) International Publication Date: 24 November 1994 (24.11.94) 



(21) International Application Number: PCT/US94/05259 

(22) Internationa] Filing Date: 12 May 1994 (12.05.94) 



(30) Priority Data: 

08/062,458 



13 May 1993 (13.05.93) 



US 



(71) Applicant: PARETO PARTNERS LTD. [US/US]; 7 Thistle 

Road, Portola Valley, CA 94028 (US). 

(72) Inventor: NAFEH, John; 7 Thistle Road, Portola Valley, CA 

94028 (US). 

(74) Agents: McGANNON, John, L. et al.; Townsend and 
Townsend Khourie and Crew, One Market Plaza, 20th 
floor, Steuart Street Tower, San Francisco, CA 94105 (US). 



(81) Designated States: CA JP, KR, European patent (AT, BE, CH, 
DE, DK, ES, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE). 



Published 

With international search report. 



(54) Title: METHOD AND APPARATUS FOR CLASSIFYING PATTERNS OF TELEVISION PROGRAMS AND COMMERCIALS 



Antennas 




22 



Discerning Device 
Features 



Pre- 
processor/ 
Feature- 
Extraction 



Video 
Audio 



Pause 



Pause 



Record 



26 



Classifier 



24 



i 



Transition Indicator 

^Activation 



Buffer 



23 



J 



Neural 
Classifier 



3 



27 

V 



T .25 

State Buffer J 



Controller 



Program 1,2,... n Decision 



VCRN 



(57) Abstract 

A method and apparatus for classifying patterns of television programs and commercials, based on learning and discerning of broadcast 
audio and video signals (12), wherein the signals might be incomplete and the number of discerning features is large. The apparatus uses a 
discerner device (10) containing feature extraction (22), an artificial neural network (27) and control mechanisms (26). The discern er device 
classifies signal patterns into classes and stores, records or displays them accordingly on different storage, recording or display devices (20) 
using the appropriate control mechanisms. The method is operable for classifying many types of signal patterns, but most importantly, 
those patterns of programs and commercials of television broadcast audio and video signals. 
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"METHOD AND APPARATUS FOR CLASSIFYING PATTERNS OF TELEVISION 
PROGRAMS AND COMMERCIALS" 



The invention herein relates to a method and 
apparatus for signal pattern classification based on 
10 information contained in audio and video television 

broadcast signals. In particular, the invention relates 
to the distinguishing of such patterns of programs and 
commercials so that certain patterns can be retained and 
used while other patterns can be discarded. 

15 

BACKGROUND Oil THE INVENTION 
Certain classifications of television broadcast 
signals have been described in several U.S. patents, 
especially to the classification of television programs 

2 0 and commercials based on transmitted broadcast signals. 

Prior work has used predetermined mechanisms to 
classify such patterns. Such mechanisms include pre-set 
timers for commercial messages. Other predetermined 
conditions to classify such patterns include transmitting 

25 signals to trigger the classification, the use of remote 

classification before transmitting, trial and error with 
recording, using multiplexed code transmitted by the 
broadcast, color versus monochrome, and fade signal 
detection among restraining mechanisms. 

30 It would be of significant value to have a 

method and apparatus capable of classifying patterns in 
general, based on audio and video signal characteristics, 
without some or all of the above mentioned pre-determined 
conditions. Classifications can be done for several 

35 distinct patterns, including television programs and 

commercials, and the classification process could be done 
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continuously. It would also be advantageous for the 
method to be similarly useful for classifying the 
patterns into classes to be stored, recorded or displayed 
in respective storage, recording or display devices. 

5 

SUMMARY OF THE INVENTION 
The present invention is a method and apparatus 
to classify patterns of television programs and 
commercials using a device which can learn and discern 
10 television broadcast audio and video signals. The 

discerning device useful in this invention contains 
signal pre-processing and feature extraction, learning 
and classification, and control mechanisms. It will also 
have adaptive mechanisms to see through noise and 
15 distortion of signals. The discerning device may be 

applied in a variety of different manners, based on the 
objects to be classified. 

The invention is intended to be useful in 
classifying many types of real-life signal patterns but, 

2 0 most importantly, programs and commercials based on 

various voice and image patterns, and generating the 
associated control commands based on the classification 
of the patterns at a continuous pace. 

The invention has the advantages of classifying 
25 and selecting what is appropriate for each class. 

BRIEF DESCRIPTION OF THE DRAWINGS 
In the drawings, Fig. 1A is a block diagram of 
the present invention to classify programs and 

3 0 commercials based on audio and video television broadcast 

signals; 

Figs. 1B-1F are detailed schematic diagrams of 
each module of the discerning device, Figs. IB and 1C 
illustrating audio and video components of the pre- 
35 processor/ feature extraction module, respectively, 



WO 94/27404 



3 



PCT/US94/05259 



10 



Figs. ID and IE illustrating a neural classifier and its 
digital representation, respectively, and Fig, ID 
illustrating the schematic representation of the 
controller; 

Fig. 2 is a block diagram showing the way to 
eliminate commercials during the recording of audio and 
video signals on the magnetic tapes of VCRs; 

Fig. 3 is a block diagram illustrating a smart 
cable control box controlled by microprocessors; 

Fig. 4 is a block diagram illustrating an 
imbedded software module in a smart cable control box, 
connected to future digital highway interconnect 
architecture . 

Fig. 5 is a block diagram illustrating a VCR as 
15 a playing device for pre-recorded programs; 

Fig. 6 is a block diagram illustrating a 
television as a receiving and displaying device for 
television broadcast signals. 

20 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

An important aspect in the method of the 
present invention is the use of an artificial neural 
network or discerning device 10 to classify signal 
patterns, and learn to modify the behavior of such 

25 patterns in response to changing environments, to produce 

consistent classifications of patterns. This neural 
network or discerning device 10 is designed in such a way 
as to learn from previous experiences, to sift through 
noise and pattern distortion, and extract the essential 

30 characteristics of each pattern from the input signal 

line 12 which often contains irrelevant information, to 
provide a classification mechanism for patterns into 
classes to be stored, recorded or displayed in different 
storage, recording, or displaying devices. Fig. 1A 

3 5 illustrates an embodiment of this invention used to 
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classify programs and commercials , and to eliminate 
commercial recordings on a VCR connected as illustrated 
in Fig, 2. 

Broadcast audio and/ or video signals are 
5 received from a variety of different sources, such as a 

satellite dish 14 , an antenna 16 , a cable 18, and/or 
storage and recording devices, so such signals have to be 
decomposed into their components by audio and video 
decoders. Those components will be detected and pre- 

10 processed in a pre-processor 22 to extract their 

essential features . 

The pre-processor output 22 will be fed to the 
classifier module 2 4 for signal discerning and pattern 
classification. Once a pattern has been classified as 

15 belonging to a certain class ^ a control signal will be 

generated and sent to the designated storage, recording 
or displaying device to record, pause or fast-forward in 
case of a VCR 20, or to cause audio and video fade in 
case of a television, as illustrated schematically in 

2 0 Fig. 2, in case the recording device is a VCR 20. 

The discerning device is made up of three 
modules: the pre-processor/ feature extraction module 22, 
the classifier module 24, and the controller module 26. 
The means of implementation in the case of the pre- 
25 processor/ feature extraction module 22 includes the use 

of analog circuits and digital logic (e.g., Digital 
Signal Processing (DSP) chips) , or a combination thereof. 
The classifier module 24 can be implemented using DSP 
chips, or more specialized chips, such as fuzzy 

3 0 controllers or other neural network-type chips. 

The preferred implementation for this invention 
is to use analog circuits implementation for the pre- 
processor/ feature extraction module 22, and the DSP 
implementation for the classifier module 24. 



35 
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Pre-Processor/Feature Extraction 

The purpose of this module 22 is to extract 
efficiently the relevant information or features for the 
classification task by pre-processing the audio and video 
5 signal output of a signal source such as a tuner or a VCR 

11 into a data stream of features. Module 22 has the 
audio frequency spectra divided into a number of bands, 
and the video is divided into a number of spacial areas 
per frame, depending on performance required. Features 
10 are measured by several parameters of the input signals, 

such as amplitude, including: minimum, maximum, mean and 
the median amplitude, and the variance and the standard 
deviation of each feature of a feature vector. The 
features and their parameters include the following: 
15 1. Audio signal: changes in power or 

amplitude over the frequency spectrum between 
program and commercial segments; 

2. Chrominance signal: changes in color 
saturation between program and commercial 

2 0 segments; 

3. Luminance signal: changes in 
brightness between program and commercial 
segments ; 

4. Vertical Interval Time Code (VITC) 
25 signal: differences in VITC transmission 

between programs and commercials; 

5. Close-Caption (CC) signal: differences 
in CC transmission between programs and 
commercials ; 

3 0 6. Color carrier jitter signal: 

transitions between programs and commercials 
due to the creation of color carrier phase 
j itter ; 
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7. Signal to Noise Ratio (SNR) signal: 
changes (SNR) between program and commercial 
segments based on their SNR. 

5 The audio feature extractor of abstractor 2 2 

provides the evaluation of the incoming summed Left and 
Right <1> audio signal from input line 12 by filtering it 
into several frequency bands, and extracting relevant 
features for each frequency band of interest, as 
10 illustrated in Fig. IB. 

Audio Processor 

The audio processor contains basic feature 
extractor circuitry which is used with each frequency 

15 band of interest. In this example, there are five bands 

28, each of which is twice center frequency of the 
previous band, i.e., 375Hz, 750Hz, 1500Hz, 3000Hz, 
6000Hz. Each band 28 is extracted using a band pass 
filter 30, with one of the center frequencies listed 

20 above. The band pass filters 3 0 are of a switched 

capacitor type, so changing the sample clock frequency 
would allow other center frequencies such as 250, 500, 
1000, 2000, and 4000Hz or 400, 800, 1600, 3200 and 
6400Hz. 

25 The feature extractor is replicated for each of 

the five frequency domains of interest. One is described 
below: 

L+R audio <1> is the input signal to the band 
pass filter along with a clock/2 signal. Clock/2 divided 

3 0 by 100 sets the center frequency of the first band pass 

filter 30. Clock/2 can be adjusted from 400KHz to 
700KHz, giving a center frequency of 4KHz to 7KHz. In 
this instance, the clock/2 signal is set for 600KHz 
yielding a 6KHz center frequency. The output of the band 

35 pass filter BPaudiol <2> 33 is RMS to DC converted, 
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giving a root mean square value of the audio signal at 
location 31, Fig. 1A. RMSaudiol <3> is fed to a pair of 
peak detectors 35 and 37, one for positive and one for 
negative, providing the maximum [MAX Bl] and minimum [MIN 
5 Bl] DC value of the RMSaudiol for the various bands of 

processor 22. A summing amplifier 36 subtracts the 
minimum from the maximum, giving a [MAX - MINI] of the 
RMSaudiol value for the frequency bands. The five MAX - 
MIN signals are fed to a s umm ing log amp 3 9 to provide 
10 the [Audio DB] signal 3 9a. 

The video feature extractor has composite video 
for its input and a multiplicity of derived signals for 
outputs. By means of appropriate filtering, including 
15 band pass filter 37 and low pass filter 3 9 and processing 

the composite video <l> signal on line 41 (Fig. 1C) is 
separated into Chrominance (C) and Luminance (Y) 
components for further processing, as illustrated in Fig. 
1C. 

20 

Chrominance Processor 

The chroma signal <2> is derived from the 
composite video input through a chrominance band pass 
filter 37a (see Fig. 1C) . A full-wave rectifier 43 and 

25 log/ amp integrator 4 5 provide the chroma RMSC <3> DC 

value. A pair of peak detectors 47 and 49, one for 
positive and one for negative, provide the maximum [MAX 
C] and minimum [MIN C] DC value of the chroma signal. 
The peak detectors sample RMSC during a sample interval, 

3 0 and are reset to zero at detector reset [DetRst] time. 

A summing amplifier 49a subtracts the minimum from the 
maximum giving a [Max - Min] of the RMS DC chroma value. 
The peak detectors and amplifier may be replicated to 
provide different picture vectors, i.e., top, middle and 
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bottom of the video frame. Each replicated group would 
have a different [sample] time interval signal. 

Chroma is also input to a gated phase-locked 
loop 51 (Fig. 1C) which compares the chroma phase with 
5 the phase of a crystal oscillator 57 running at chroma 

frequency. The phase difference PhsDif <4> on line 53 
signal is low-pass filtered to derive phase control 
signal, PhsCtrl <5> 55, and fed back to the VCO input of 
the PLL. PhsDif is further processed by a high-pass 
10 filter 59. The high-pass filter detects significant 

changes in chroma relative the oscillator. PhsCtrl is 
processed by an integrator to derive the chroma phase 
state [Phase C] . 

Luminance Processor 

Composite video is passed through a chroma trap 
61 and low-pass filtered by filter 63 to derive Luma <6>. 
Luma is integrated by integrator 65 to provide the 
average DC value AvgY <7>. A pair of peak detectors 67 
and 69 f one for positive and one for negative, provide 
the maximum [MAX Y] and minimum [MIN Y] DC value of the 
luma signal. The peak detectors sample AvgY during 
sample interval, and are reset to zero at detector reset 
[DetRst] time. A summing amplifier 71 subtracts the 
minimum from the maximum giving a [Max - Min] of the 
average luma value. The peak detectors and amplifier may 
be replicated to provide different picture vectors, i.e., 
top, middle and bottom of the video frame. Each 
replicated group would have a different [Sample] time 
interval signal. 

Svnc Processor/Timing Generator 

Luma is stripped of all but synchronizing 
signals in the sync stripper 7 3 (Fig. 1C) . Composite 
35 sync CmpSync <8> is fed to the H&V detector 75 which 
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provides the system timing signals [V Sync] and [H Sync] 
on lines 8 0 and 81. H Sync and V Sync are inputs to the 
timing generator which counts V and H sync pulses to 
derive the appropriate time intervals for resetting the 
5 detectors and integrators [DetRst] for sampling the 

chroma and luma information [Sample], for detecting 
Vertical Interval Time Code data [VITC time], and Closed 
Caption data [CC time]. 

VITC Time on line 82 enables a comparator 83 to 
10 look for VITC data and, if detected, outputs [VITC 

Active] on line 84. cc Time on line gates comparator 83 
to look for Closed Caption data and, if detected, outputs 
[CC Active] on line 86. 

15 Classifier 

The input to the network classifier consists of 
a sequence of feature vectors extracted using the pre- 
processor/feature extraction module as described above. 
These feature vectors contain information about 

20 chroma/ lumina, audio spectrum, VITC, close-caption, color 

carrier jitter, and signal-to-noise ratio data. In all, 
each feature vector contains k indicators (approximately 
60-100 analog indicators) scaled to a range of +1 to -l. 
The feature vector is generally defined as the vector 

25 F(t). 

These features are sampled, analog to digital 
converted and passed to the input of the network at a 
rate of 1-60 per second, depending on the speed required 
for pattern classification. The total input to the 

3 0 network classifier consists of the buffered sequence 

F(t), F(t-s), F(t-2s), ...F(t-ns), where s is the 
sampling interval in seconds, and n is the number of 
samples. Thus, the network sees the last (n+1) feature 
vectors representing a window of several seconds. The 

3 5 total number of inputs to the network is equal to the 
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number of samples (n+1) , multiplied by the number of 
indicators k in the feature vectors. An additional input 
is the current state of the system: +1 if the apparatus 
is currently recording a program, and -1 if no recording 
5 is taking place. This additional input may be thought of 

as the previous decision of the classifier made at the 
previous transition (or, instead, several previous 
decisions) . 

A mult i- layer neural network is shown in Fig. 

1° ID- It is comprised of a layered arrangement of 

artificial neurons in which each neuron of a given layer 
feeds all neurons of the next layer. The inputs to a 
neuron are multiplied by various coefficients called 
weights, which represent the synaptic connectivity 

15 between a real neuron in biology in a specific layer and 

another neuron in the following layer. All weighted 
inputs to a neuron are added, and the result is 
"compressed" by a sigmoid function (e.g., Arctanh) , as 
illustrated in Fig. ID. 

2 0 Training of the artificial neural network is 

accomplished using the backpropagation algorithm in which 
a set of inputs are associated with a set of desired 
output, and the weights are iteratively adapted to 
achieve the desired mapping. In the classifier, the 
25 value of an output node converges to the probability of 

being in a class given the input. 

The network will consist of multiple layers 
(e.g., three layers as in Fig. ID) of synaptic weights 
and of several hundreds to several thousands of inputs 

3 0 feeding hidden neurons, feeding to 1 output (the number 

of inputs and hidden neurons may vary slightly, as they 
will be optimized during training) .The single output of 
the network is used to make a decision as to whether the 
broadcast is either a commercial or a program, following 
3 5 a detected transition. The output neuron level indicates 
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the posterior probability of the broadcast being a 
program given the input minus the posterior probability 
of the broadcast being a commercial given the input. 

Ideally , the output should equal +1 for a 
detected program, and -l for a detected commercial. A 
threshold of zero is used to make a decision as to 
whether a program or a commercial is indicated. (Other 
thresholds may be used to adjust the risk associated with 
making incorrect decisions. For example, it is less 
serious to make the mistake of recording a commercial 
than it is to make the mistake of not recording a 
program.) The final commercial/program decision is made 
based on the consistency of the network output for 
several consecutive clock cycles. This is necessary to 
avoid spurious decisions immediately following a 
transition. The final decision converts the decisional 
index into a program index. 



In the preferred implementation, the Digital 
Signal Processor (DSP) based classifier has as its input 
the feature signals or vectors from the feature 
extractor, and the parameter values from the EEPROM (or 
PROM) memory. Its output is a signal, classifying the 
feature contents as commercial or program. It provides 
this functionality by controlling a multiplexed analog to 
digital converter to gain digital equivalents of the 
feature vectors, and then processing them in accordance 
with the parameter values held in external memory. 

The digital neural classifier presented in Fig. 
ID is implemented using DSP chips, as illustrated in Fig. 
IE. All indicators of the feature vector audio, video 
and state) are input to the DSP memory (e.g., Data RAM) 
over the data bus, as illustrated in Fig. IE. Only audio 
and video feature vector indicators are fed through the 
analog-to-digital (A/D) converter. All indicator values 
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are then multiplied by the appropriate weights, and added 
together in groups corresponding to neurons in the hidden 
layer, and finally "compressed" with a sigmoid function 
(e.g., Arctanh) . The results of these computations are 
5 the output values of all the neurons on a layer. This 

same process is repeated for each hidden layer. The last 
layer is the output layer that produces the value in the 
state/ logic buffer, as illustrated in Fig. IE. 

The multiplication of a weight is accomplished 

10 by using the 16 x 16 multiplier, and the additions are 

carried out by the 3 2 -bit accumulator 90, as illustrated 
in Fig. IE. The weights are stored in the parameter 
memory (e.g., EEPROM or ROM). 

The central logic (controller ext. I/O) 91 with 

15 the aid of the stack 92, program data memory 93, program 

control 94, 32-bit ALU 90, memory control logic 95, and 
shifters 9 6 coordinate all these activities: inputs, 
neural computations, and outputs, as illustrated in Fig. 
IE. In higher performance implementations, more of these 

20 activities will occur concurrently, in order to make 

optimized use of resources. 

Controller 

The control used in this invention includes 
25 several remote control protocols that are in commercial 

use to control a large number of consumer and 
professional VCRs. A schematic representation of the 
controller is shown in Fig. IF. 

The controller provides the interface between 
30 the external devices and the internal logic. The 

controller has as inputs the [Program Index] signal from 
the classifier, and the user selected mode control 
inputs, such as [View], [Record], [Skip] and [PIP], where 
PIP stands for Picture-in-Picture , which is available on 
35 certain televisions. 
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Contr oller Interface 

The control encoder provides the logic to 
control the fader, PIP and VCR operation modes. The 
serializer encodes the VCR control signals to a format 
5 suitable for controlling a VCR, using the IR remote 

control interfaces . 

If the mode selected is [View] , then the 
program index signal causes the fader to be inactive. 
During commercial periods, the fader causes the [L audio 
10 and R audio out] and [video out] to be attenuated by 3 or 

6 DB depending on the setting of the fader switch. If 
[Record] mode is selected, the program index signal 
activates the record signal sent by the controller to the 
appropriate VCR. During commercials, the pause signal is 
15 active. In [Skip] mode, the 'play signal is activated 

during a program, and Fast Forward is activated during a 
commercial. In [PIP] mode during a commercial, the 
current channel image and sound get relegated to the 
small picture. 

20 A variety of configurations can be connected 

together depending on the intended use. The pattern 
classification and controlling of the storage, recording 
or displaying devices can be formed into a wide variety 
of configurations, including different varieties of audio 

25 and/ or video sources, receiving devices, and recording, 

storage and displaying devices. The various 
configurations may be formed by connecting different 
combinations of audio and/or video signal sources, 
receiving devices, and storage, recording or display 

3 0 devices. 

Fig. 2 schematically illustrates the invention 
with a VCR as a receiving and recording device. 

Fig. 3 is a block diagram showing the use of 
the present invention with a smart cable control box 
3 5 (microprocessor) . 
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Fig. 4 is a block diagram showing the use of 
the present invention with a future digital highway 
interconnected architecture. 

Fig. 5 is a block diagram showing the use of 
5 the present invention with a VCR as a playing device for 

pre-recorded programs that include commercials. 

Fig. 6 is a block diagram showing the use of 
the present invention with a VCR as a receiving device 
and a television as a displaying device. 
10 Performance of the method of the present 

invention will depend on the degree with which the 
discerning device will be able to discriminate between 
different classes of patterns, as well as the number of 
classes the device is required to classify among all 
15 classes. 

While the invention has been described above 
primarily in the context of classifying patterns of 
programs and commercials based on audio and video 
television broadcast signals, this should be considered 
20 to be merely an illustration of a typical use of the 

method . 

It will be evident to those skilled in the art 
that there are numerous embodiments of this invention 
which, while not expressly set forth above, are clearly 
25 within the scope and spirit of the invention. Therefore, 

the above description is to be considered exemplary only, 
and the actual scope of the invention is to be defined 
solely by the appended claims. 
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WHAT IS CLAIMED IS ! 

1. Apparatus for processing electromagnetic 
signals capable of being separated into a first group and 
a second group comprising: 

means for receiving said signals as a signal 

stream ; 

means coupled with the receiving means for 
forming the signal stream into segments which can be 
represented as parameters, with the groups being mixed 
together; 

means coupled with said forming means for 
storing the signal parameters after they have been 
f ormed ; 

15 means coupled with 'said storing means for 

classifying the segments into the first and second 
groups, said classifying means being operable to generate 
at least one decisional index representative of at least 
one signal segment and their characteristic of at least 

20 one of the said groups of signals; and 

a signal controller coupled with the 
classifying means for receiving and being responsive to a 
decisional index therefrom, said controller being 
operable to control a signal sensitive structure adapted 

25 for directing said groups of signals to respective end 

use locations. 

2. Apparatus for processing signals as set 
forth in claim 1, wherein said first group of signals 
includes program signals and the second group of signals 
includes commercial signals. 



30 



3. Apparatus as set forth in claim 1, wherein 
the means for classifying the segments includes a neural 
3 5 network. 
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4. Apparatus as set forth in claim 1, wherein 
is included means defining a transition indicator for 
coupling the storage means to the classifying means , said 
transition indicators being operable for actuating the 

5 classifying means to cause a decision to be made by the 

classifying means as to the classification of at least 
one signal segment, relative to said first and second 
groups . 

5. Apparatus as set forth in claim 1, wherein 
is included means for storing past decisions, and means 
for coupling the last-mentioned storing means to the 
classifying means to allow such past decisions to be 
included in the decisions to determine which signal 
groups the signal segments belong to. 

6. Apparatus as set forth in claim 5, wherein 
is included a transition indicator means for coupling the 
storage means to the classifying means, said transition 

2 0 indicator being operable for actuating the classifying 

means to cause decision to be made by the classifying of 
each segment relative to the first and second groups, 

7. Apparatus for processing electromagnetic 
25 signals representing program and commercial signals 

comprising: 

means adapted to be coupled to a source of said 
signals for receiving the signals as a signal stream; 

means coupled with the receiving means for 

3 0 forming the signal stream into segments which can be 

represented as parameters, with the groups being mixed 
together ; 

means coupled with the forming means for 
storing the signal parameters after they have been 
3 5 formed; 
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a neural network coupled with said storing 
means, said classifying means being operable to generate 
at least one decisional index representative of at least 
one signal segment and their characteristic of at least 
5 one of said groups of signals; 

means defining a transition indicator for 
coupling the storage means to the classifying means, said 
transition indicator being operable for actuating the 
neural network to cause a decision to be made by the 
10 neural network as to the classification of at least one 

signal segment, relative to said first and second groups; 

means for storing past decisions; 

means for coupling the last-mentioned storing 
means to the classifying means to allow such past 
15 decisions to be included in the decisions to determine 

which of said signal groups the signal segments belong 
to ; and 

a signal controller coupled with the neural 
network for receiving and being responsive to a 
20 decisional index from the neural network, said controller 

being operable to control a signal sensitive structure 
adapted for directing said groups of signals to 
respective end use locations. 

25 8. Apparatus as set forth in claim 7, wherein 

said electromagnetic signals include audio signals, 
chrominance signals, luminance signals, vertical integral 
time code (VITC) signal, a close-captioned (CC) code 
signal, a color carrier jitter (CCJ) signal, and a signal 

3 0 to noise ratio (SNR) signal. 

9. Apparatus as set forth in claim 7, wherein 
the video signals are divided into a number of bands, 
each band being the source of signals from which 
3 5 characteristic numbers are determined. 
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10. Apparatus as set forth in claim 7, wherein 
said audio signals are stereo signals. 

11. Apparatus as set forth in claim 7, wherein 
5 the chrominance and luminance signals are derived from 

the video signals and divided a number of spatial bands 
from which several characteristic numbers can be 
determined . 

10 12. A method for processing electromagnetic 

signals capable of being separated into a first group and 
a second group comprising: 

receiving said signals at a location as a 
signal stream; 

15 forming the signal "stream into segments which 

can be represented as parameters, with the groups being 
mixed together; 

storing the signal parameters after they have 
been formed; 

20 classifying at least one segment into the first 

and second groups; 

generating as a function of said classifying 
step at least one decisional index representative of at 
least one signal segment and their characteristic of at 
25 least one of the said groups of signals; and 

in response to a decisional index, controlling 
a signal-sensitive structure adapted for directing said 
groups of signals to respective end use locations. 



30 



13. A method as set forth in claim 12 , wherein 
said first group of signals includes program signals and 
the second group of signals includes commercial signals. 
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14. A method as set forth in claim 1, wherein 
the classifying step includes operating at least one 
neural network • 



5 15. A method as set forth in claim 1, wherein 

is included the step of sensing a transition in at least 
one signal segment from one said group to another group; 
and 

actuating the classifying means to cause a 
10 decision to be made by the classifying means as to the 

classification of at least one signal segment, relative 
to said first and second groups. 



16. A method as set forth in claim 12 , wherein 
15 is included storing past decisions, and allowing such 

past decisions to be included in the decisions to 
determine which signal groups the signal segments belong 
to. 



20 17. A method for processing electromagnetic 

signals representing program and commercial television 
signals comprising: 

receiving the signals as a signal stream at a 
first location; 

25 forming the signal stream into segments which 

can be represented as parameters, with the groups being 
mixed together; 

storing the signal parameters after they have 
been formed; 

3 0 operating a neural network after the storing 

step to classify at least one signal segment and to 
generate at least one decisional index representative of 
at least one signal segment and their characteristic of 
at least one of said groups of signals; 
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sensing a transition from one group to the 
other group by actuating the neural network to cause a 
decision to be made by the neural network as to the 
classification of at least one signal segment, relative 
5 to said first and second groups; 

storing past decisions; 

allowing such past decisions to be included in 
the decisions to determine which of said signal groups 
the signal segments belong to; and 
10 controlling a signal-sensitive structure for 

directing said groups of signals to respective end use 
locations. 

18. A method as set forth in claim 17, wherein 
said electromagnetic signals 'include audio signals, 
chrominance signals, luminance signals, vertical integral 
time code (VITC) signal, a close-captioned (CC) code 
signal, a color carrier jitter (CCJ) signal, and a signal 
to noise ratio (SNR) signal. 

19. Apparatus as set forth in claim 17, 
wherein the video signals are divided into a number of 
bands, each band being the source of signals from which 
characteristic numbers are determined. 

20. Apparatus as set forth in claim 17, 
wherein said audio signals are stereo signals. 

21. Apparatus as set forth in claim 17, 

3 0 wherein the chrominance and luminance signals are derived 

from the video signals and divided into a number of 
spatial bands from which several characteristic numbers 
can be determined. 



15 



20 



25 



WO 94/27404 



1 / 1 1 



PCT/US94/05259 




WO 94/27404 



2 / 1 1 



PCT/US94/05259 




WO 94/27404 



3 / 1 1 



PCT/US94/05259 



o 
S 



co 



1 1 



o 



O 




i It 



o 

cd 

CD 
£1 
CO 

O 



0) 



x P ® 

■i « S 

Sco Q 




A 
cvj 

V 
CO 

E 
o 



t 



O 

CD 
CO 
CO 



O 

a> 
Q 

a> o 
CL X 



CO 

DC 
"co 
Q 

cr> 



x 

CO 



cd 
c3 



CD 

1_ 



A i 



>- 



>- 




CO a< 
X p * 



co 
if) 
CO 

cl 52 
if Sf 



A 

b 

CD 
CO 
CO 



CD 

r CO CD 
— O 



CD 

"8 



CD I 



4 



a 
co 

O 

CO 

E 



■ 1 1 10 
I CO 

iri\ I I — ' — I 



CD -J 

IS-! 

(3D. 



CL CD 
O = 



I 



CO 

DC 
"co 
Q 



A 

m 
_y 

O 

CD 
CO 
CO 



a 

aj 

E 

co 
CO 



4 



rfi 

V 

CD 



CD 
CO 



o 
co 



00 



CO q5 

x E «S* 



CD 



IT) 

to 



cU 
E 
co 
CO 



co 
DC 

CO 

Q 



CD 

co 



V 0 
c 

I 



A 
CO 
V 



10 



A 1 



o 
c 
>■> 
CO 



o 
00 

VJ 



CD 
.> 

o 

< 

O 
O 

CD- 
CO 



CD 
> 

< 

O 



00 



J 



CO 
00 



p , 

S E 2 

CO O m 

a o cL 



> ts 

"\ cd 



12 co 

CZ CO CD 

co co s 

CO 0- ix. 



CO 
CD 



CO 
CO 



"co 

E % A 

8 > v 



8£ 

O COm 
— I CL LL 



A 
CD 

X 

E 
3 



CO- 
Ca. 

E 
o 
O 



CO 



I 



coco 



I 



CVJ 

00 



CD 

E 
I- 

O 
O 



in 
co 



CL 

E 
co 
CO 



C C CD 

o .E o> 
OhO 



o 

CD 



CO CL 



Hi' 



CD 



U / 1 1 

WO 94/27404 PCT/US94/05259 




8> £ 

3 3 ^ o ® 

CD CD O ■— 3 

LL > i I ID 



WO 94/27404 



5 / 1 1 



PCT/US94/05259 



if) 

iS 
Q 




o c5 a> 

3 'g* 1 

CO —I CD 



s 

> 

15 

O 
T5 



(0 

CD 
> 
CD 

CO 

O 

CD 
"O 

> 



2 ^2 

<5 ^ O 

E o en 

« E Q_ 



ill 



CO 

a> 

i= "c7T 
E | 

ct a 



CD 





























CO 


"c 








to 


ZJ 

o 








Q 


O 


.2? 


O 




E 


E 


e 

c 


J* 


2? 


J? 




o 


o> 


CD 


o 




Sta 


e 


o 


O 


LU 


Dl 


0- 



to 

IT II 



CD 

X 

CD 



O 

cn 

CO 
CD 

o 
p 



CO 

c 
g> 

CO 
b 



o 



Z> 
_J 
< 

JO 

CM 
CO 

Z7 



o 
o 
< 



CVJ 
CO 



2? 

CD 

1c 
CO 



< 

CO 

"co 

Q 



CD 

CD 



~7 



LU 



WO 94/27404 6 / 1 1 PCT/US94/05259 



-3 
O 
o 

CD 

■g 
> 



3 
O 
g 











cd 








i 








CO 




a 




CD 




k_ 




O 




CO 



"8 

CD 
Q 
CO 

o 

CO 



13 

O 
g 

GC 



I 



a> 

s 

m 

Q 

CO 

o 

CO 



s 



a> — 

o = 2 

EE E 2 c 

O ® to o 

> H Q. O 



O 


"B 


(/) 


o 


z> 


o 


CO 


a> 


Ql 


CC 



or 



CD — 

o 2 

DC E c 

O CD O 

> a: o 



O 

55 



rrrrri 



CD 
CO 
ZJ 
CO 
CL 



CD 

15 



QJ 

O 

co 



CD 

*£= 
CD 
CO 



o c 

O LU 



o 

CD 

■g 
> 

Ol 

E 
o 
O 



g 

3 



g 

< 



CD 

> 

CD 

E 



g. 

CO 



CO 

CD \ 
or \ 



o 

LL. 



i i i 



GO 
Q 

CO 



CD 

a 

CO 



a> 

CD 
CO 
CD 

E 



Hi- 



.9? 
"w 

CO 

O 

E 
o 



co 

CD 

E 
2 

fa 



g 

CD 



WO 94/27404 



7 / 1 1 



PCT/US94/05259 



O) 




c 










o 


isc 


evi 







O S CD 
O D. Q 




CO o 
Q. O 



c 

CD 



CM 

d 

LL 



hi 

CO 

O 



WO 94/27404 



PCT/US94/05259 

8 / 1 1 




WO 94/27404 



9 / 1 1 



PCT/US94/05259 



T3 ~ 
a> </) 

£ w 
c a> 

I E 5 

ill 

5 o § 



. o 
o i= 

-S § 

>o 

3 CO 

< Q 




O — Cu 
U.O 



o 

si 2 

QX O 



WO 94/27404 



10/11 



PCT/US94/05259 



55 
"p 




WO 94/27404 



11/11 



PCT/US94/05259 




INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US94/05259 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(5) :H04N 5/76, 5/91, 5/222 

US CL :348/571, 907, 722; 358/908, 335; 360/69; 395/22 
According to International Patent Classification (IPC) or to both national classification and IPC 

B. FIELDS SEARCHED 

Minimum documentation searched (classification system followed by classification symbols) 
U.S. Please See Extra Sheet. 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 
Please See Extra Sheet. 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


A 


US, A, 4,504,851 (JANKO et al.) 12 March 1985, col. 2, 
lines 19-68. 


1,7,12,17 


X 
Y 


US, A, 4,750,213 (NOVAK) 07 June 1988, see whole 
document. 


1 ,2,4-6,1 2, 
13,15,16 






3,7,8,10,14,17 
,18,20 


A 


US, A, 5,103,431 (FREEMAN et al.) 07 April 1992, see 
abstract. 


1,7,12,17 


A 


US, A, 5,146,541 (SPEIDEL) 08 September 1 992, see whole 
document. 


1,7,12,17 


Y 


US, A, 5.161,014 (PEARSON etal.) 03 November 1 992, see 
whole document. ' 


3,7,8,10,14,17 
,18,20 



X Further documents are listed in the continuation of Box C. [ _J| See patent family annex. 



•A- 

■E" 
■L" 

•O" 
. p . 



Special categories of cited document*: 

document defining the general state of the art which it not considered 
to be part of particular relevance 

earlier document published on or after the international filing date 

document which may throw doubts on priority claim(s) or which is 
cited to establish the publication date of another citation or other 
i (as specified) 



document referring to an oral disclosure, use, exhibition or other 
document published prior to the international filing date but later than 



later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand the 
principle or theory underlying the invention 

document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive step 
when the document is taken alone 

document of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 
12 AUGUST 1994 


Date of mailing of the international search report 

SEP 0 8 1994 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703) 305-3230 


Auth^ 

/Si 
Telep 


Pgged officej^ J\^_^J^C^ 

&ET METJAHIC 

hone No. (703) 305-4741 



Form PCT/1SA/210 (second sheet)(July 1992)* 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US94/05259 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


A 
A 


US, A, 5,165,069 (VITT et al.) 17 November 1992, see FIG. 1 
and corresponding text. 

Giarratano, Joseph C. et al., "FUTURE IMPACTS OF 
ARTIFICIAL NEURAL SYSTEMS ON INDUSTRY", ISA 
Transactions, Vol. 29, No. 1, pp. 9-14, 1990. 


1,7,12,17 
1,7,12,17 



Form PCT/ISA/210 (continuation of second sheet)(July 1992)* 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US94/05259 



B. FIELDS SEARCHED 
Minimum documentation searched 
Classification System: U.S. 

US CL: 348/571, 907, 722, 5, 180; 358/908, 335, 139, 10, 185, 160, 310-312; 360/69, 27, 14.1, 33.1, 74.1, 71; 

395/22, 21; 455/67. 

IPC(5): H04N 5/76, 5/91, 5/222 

B. FIELDS SEARCHED 

Electronic data bases consulted (Name of data base and where practicable terms used): 
APS 

search terms: (commercial^ or advertisement) and (record? or vcr or vtr or video(w)(tape or cassette)(w)recorder) and 
(neural or neuron) and audio and video and memory 



Form PCT/1SA/210 (extra sheet)(Ju!y 1992)* 



